Method for executing virtual application delivery controllers having different application versions over a computing device

ABSTRACT

A method for executing virtual application delivery controllers (vADCs) having different application versions over a computing device. The method comprises installing a virtualization infrastructure in the computing device; creating by the virtualization infrastructure a plurality of vADCs having different application versions, wherein each vADC is created from a software image maintained in a hardware infrastructure of the computing device; gathering version information associated with each of the plurality of vADCs; independently executing the plurality of vADCs over an operating system of the computing device; and controlling the execution of the plurality of the vADCs over an operating system of the computing device using the virtualization infrastructure using in part the version information. In one embodiment, each of the plurality of vADCs does not execute its own guest operating system.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. provisional application No.61/448,472 filed on Mar. 2, 2011, the contents of which are hereinincorporated by reference.

TECHNICAL FIELD

This invention generally relates to virtualizations of applicationdelivery controllers (ADCs).

BACKGROUND

An application delivery controller (ADC) is a network device installedin a datacenter or multi-datacenter system to remove load from webservers in the datacenter. That is, an ADC typically distributesclients' requests between the web servers in a datacenter to balance theload. In a multi-datacenter system, an ADC is deployed in eachdatacenter to redirect clients' requests to a datacenter that would bestserve such requests. Typically, the redirection decision is based on thelocation of the client from the datacenter. The ADC is a network deviceand, as such, includes computing resources, such as memory, one or morecentral processing units (CPU), storage, network connectivity, and soon.

A virtual machine (VM) is a software implementation of a computer thatexecutes programs like a physical machine. The virtualization technologydecouples the hardware from the software, thus allows the sharing of theunderlying physical hardware resources between different virtualmachines, each running its own operating system (guest). Thus, thevirtualization, which is typically performed by a hypervisor, allowsmultiple operating systems to run concurrently on a host computer. Thehypervisor presents the guest operating systems with a virtual operatingplatform and monitors the execution of the guest operating systems.Further, the hypervisor defines the allocation of resources (e.g., CPUpower, memory, network bandwidth, etc.) for each guest operating system.

Virtualization of an ADC device can improve the performance ofdatacenters and reduce costs and overhead to the service providers.Similar to any other data center application, the ADC devices ofdifferent customers or applications can be consolidated as multiplevirtual ADC instances running on a single hardware device. Astraightforward approach to achieve this process would be to run aconventional hypervisor to control one or more ADC virtual machines.However, conventional hypervisors are primarily designed to supportvirtualization of general purpose computing devices, e.g., servers, andnot network devices, such as ADCs. Network elements are measured bytheir high forwarding capacity and low latency, unlike serverapplications that are measured by their capacity of CPU intensive taskprocessing. For example, conventional virtualization solutions cannotguarantee low latency when processing data packets, and further, theirthroughput and capacity is limited as only a small number of virtualmachines can be executed on a physical computing device.

To improve the utilization of ADC resources, multiple ADC virtualmachines should be independently executed in a signal device. Thestraightforward approach is that the hypervisor executes a guestoperating system within an ADC virtual machine created by thehypervisor. Thus, the ADC virtual machine runs its own operating system.Further, the hypervisor creates and controls by the virtual machines.Such an approach may limit the independency of the ADC virtual machinesexecuted by the hypervisor and their performances.

Therefore, the straightforward and conventional virtualization solutionsare not optimized to support efficient virtualization of ADCs.

SUMMARY

Certain embodiments disclosed herein include a method for executingvirtual application delivery controllers (vADCs) having differentapplication versions over a computing device. The method comprisesinstalling a virtualization infrastructure in the computing device;creating by the virtualization infrastructure a plurality of vADCshaving different application versions, wherein each vADC is created froma software image maintained in a hardware infrastructure of thecomputing device; gathering version information associated with each ofthe plurality of vADCs; independently executing the plurality of vADCsover an operating system of the computing device; and controlling theexecution of the plurality of the vADCs over an operating system of thecomputing device using the virtualization infrastructure using in partthe version information. In one embodiment, each of the plurality ofvADCs does not execute its own guest operating system.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments are particularly pointed out and distinctly claimedin the claims at the conclusion of the specification. The foregoing andother objects, features, and advantages of the invention will beapparent from the following detailed description taken in conjunctionwith the accompanying drawings.

FIG. 1 is a block diagram of a virtualized ADC device according to anembodiment of the invention;

FIG. 2 is a diagram illustrating the execution of a vADC on multiplecores;

FIG. 3 is a diagram of a multi-blade system utilized to describe anembodiment of the invention;

FIG. 4 is a flowchart illustrating the process for creating a vADCaccording to an embodiment of the invention;

FIG. 5 is a diagram illustrating the process of allocating capacityunits to a created vADC;

FIG. 6 is a diagram illustrating the scheduling process according to anembodiment of the invention;

FIG. 7 is a flowchart illustrating a vADC selection process performedaccording to an embodiment of the invention; and

FIG. 8 is a flowchart illustrating the process for routing packetsprocessed by a vADC according to an embodiment of the invention.

FIG. 9 is a diagram utilized to describe a process for independentlyexecuting vADCs having different application versions.

FIG. 10 is a diagram utilized to describe a process for enablingsoftware upgrades of vADCs.

FIG. 11 is a flowchart illustrating a method for independently executingvADCs over a virtualized ADC device.

DETAILED DESCRIPTION

The embodiments disclosed herein are only examples of the many possibleadvantageous uses and implementations of the innovative teachingspresented herein. In general, statements made in the specification ofthe present application do not necessarily limit any of the variousclaimed inventions. Moreover, some statements may apply to someinventive features but not to others. In general, unless otherwiseindicated, singular elements may be in plural and vice versa with noloss of generality. In the drawings, like numerals refer to like partsthrough several views.

FIG. 1 is an exemplary and non-limiting diagram illustrating thearchitecture of a virtualized ADC device 100. According to thetechniques disclosed herein a virtualized ADC device is a physicalcomputing device that can execute a plurality of instances of virtualADCs (hereinafter vADC). The physical computing device may include, forexample, a server, a blade server, a blade system, a multi-blade device,and a network device, such as a load balancer, an application deliverycontroller, and the like.

As illustrated in FIG. 1, the virtualized ADC device 100 includes ahardware layer 110 that comprises the computing resources of the device100. The computing resources include, but are not limited to, one ormore central processing units (CPUs), each CPU with one or moreprocessing cores, a system memory, a storage disk, a system board, amemory management unit, registers, network ports, and so on. A singleoperating system (OS) is executed over the hardware layer 110. The OSmay be, but is not limited to, a Windows-based operating system, a Linuxbased operating system, a UNIX based operating system, VXwork6, and thelike.

The computing resources of the hardware layer 110 are managed by amanagement module 120. Specifically, the management module 120 sets thevarious components of the hardware layer 110, defines network parametersand addresses, supervises the allocation of device resources, creates,and halts vADC processes. A user (e.g., a system administrator) canaccess the management module 120 using, for example, a Web-basedapplication, a SNMP-based application, a command-line interface (CLI),web services API, and the like.

In the virtualized ADC device 100, one or more vADCs 130-1 through 130-nare created and executed. Each of the vADC 130-1 through 130-n is aprocess that acts logically as a physical ADC device. That is, each vADC130-i (i=1,2, . . . ,n) performs the tasks of a physical ADC device.These tasks include, but are not limited to, load balancing of traffic,traffic acceleration, traffic compression, traffic caching, SSLoffloading, and so on. Each vADC 130-i has a separate networkingconfiguration, a MAC address, and an application delivery configuration.It should be noted that a vADC 130-i process does not execute its ownguest operating system, as conventionally performed by virtual machines.The traffic distributor 140 directs incoming traffic to one or more ofthe vADCs 130 and routes traffic between the vADCs 130. The trafficdistributor 140 also schedules the execution of the vADC on the CPUcores of the device 100.

In accordance with an embodiment, the management module 120, vADCs 130,and traffic distributor 140, communicate using a virtualizationcommunication protocol (VCP). The VCP provides software independency ofthe vADCs 130-1 through 130-n from the underling hardware layer 110 andits operating system. This allows independently executing multiple vADCs130-1 through 130-n, where each vADC may be of a different version andexecution logic. As a result, service providers can choose whichapplication version and which vADC to run. The service providers canalso choose which vADC to upgrade, thus enabling partial system upgrades(software upgrade). It should be appreciated that a partial systemupgrade minimizes the risks of introducing new software versions andreduces the overall downtime of a datacenter.

Each of the plurality of vADCs 130 is independently managed. That is,each vADC manages its own configuration, user privileges, alerts, andreports. This allows, for example, setting different privileges todifferent vADCs running on a virtualized vADC device. The variousembodiments for executing different vADCs in a virtualized ADC deviceare discussed in greater detail below.

In accordance with an embodiment, a vADC can be executed on a singlecore, a part of a core, or multiple cores of one or more CPUs. Asillustrated in FIG. 2, each of CPUs 210 and 220 includes 4 cores, 210-1through 210-4 and 220-1 through 220-4. The CPUs are part of the hardwarelayer of the virtualized ADC device. A vADC 130-i can run on anycombination of cores 210-1 through 210-4 and 220-1 through 220-4. In theexample shown in FIG. 2, a vADC 130 runs on cores 210-1 and 210-2 of theCPU 210 as well as cores 220-3 and 220-4 of the CPU 220. The selectionof CPU cores is described in detail below.

Referring back to FIG. 1, a vADC 130-i is exposed to the network as aseparate network device set with its own set of virtual internal ports135. However, data traffic of a vADC 130-i is actually sent and receivedthrough the physical external ports 115 of the virtualized ADC device100. Each virtual internal port 135 is set with a unique media accesscontrol (MAC) address. The number of virtual internal ports 135 assignedto each vADC 130 is bounded by the number of external ports 115. Thatis, the number of virtual internal ports 135 of a vADC 130-i cannotexceed the number of the external ports 115.

In accordance with certain embodiments, association between internalports and external ports may be performed using either a dedicated porttopology or shared port topology. The dedicated port topology assumesfull network infrastructure separation between vADCs 130 running on thesame virtualized ADC device 100. That is, a specific external port 115is associated with an internal port 135 of a single specific vADC 130.In a shared port topology, two or more different vADCs 130-1 through130-n can share the same external port 115, where traffic separation isperformed according to VLAN association and destination MAC addresses.

The traffic which flows within the virtualized ADC device 100 can belogically divided into two groups: external traffic, i.e., trafficreceived through the external ports 115, and internal traffic, i.e.,traffic generated by the virtualized ADC's components, i.e., the vADCs130, the management module 120, the hardware layer 110, the trafficdistributor 140, and so on. Primarily, external traffic should beload-balanced. The internal traffic carries configuration updates, eventnotification, and statistical information.

Incoming external traffic, received through one of the external ports115, is sent to one of the vADCs 130 selected by the traffic distributor140. Specifically, first, a packet arrives to the virtualized ADC device100, through one or more external ports 115, and is processed by thehardware layer 110 (e.g., an Ethernet MAC controller). Then, processedpackets are sent to the traffic distributor 140 that selects a vADC130-i for processing the packets according to one or more of the VLANtags, MAC addresses, or any other layer-2 parameters designated in thepackets and a vADC selection process. The traffic distributor 140 mayalso select between one or more cores (either physical or logical CPUcores) within the selected vADC 130 based, in part, on layer 3 or layer4 (TCP/IP) parameters of the OSI model. The vADC selection process isdescribed in detail below.

The traffic distributor 140 also checks if an incoming packet should becloned and sent to multiple destinations. For example, if an incomingpacket is a broadcast packet or has an unknown destination MAC address,the traffic distributor 140 clones the packet and sends copies of thesame packet, to multiple vADCs 130 on the same layer 2 VLAN that thatthe packet arrived from. Finally, packets received at a vADC 130-i (i=1,2, . . . , n) are processed to perform the tasks defined for the vADC130-i. Such tasks may include, for example, distributing packets betweenweb servers, or any other ADC's functions.

In the transmit direction, packets processed by a vADC 130-i arereturned to the traffic distributor 140, which determines the finaldestination of each outgoing packet. Packets that should be sent outsideof the virtualized ADC device 100 are forwarded to the hardware layer110 which transmits the packets through the external ports 115 to itsdestination. The destination external port is determined according to,for example, layer-2 parameters (e.g., destination MAC address) of thepackets or any other indicator. Packets can also be sent from a vADC130-i to an internal destination port or ports for processing by othervADC or vADCs 130-i. The processes for traffic distribution from/tovADCs and between vADCs are described below with reference to FIGS. 7and 8.

FIG. 3 schematically illustrates a multi-blade system 300 that includesa plurality of blade servers 310-1 through 310-M and a switch 320.According to certain embodiments of the invention, vADCs can be executedon the multi-blade system 300. The switch 320 determines to which bladeserver 310-j (j=1, 2, . . . , M) to distribute an incoming traffic. Inaccordance with one embodiment, ADC virtualization may be performed on aset of blade servers including one or more of the blade servers 310-1through 310-M, where the switch 320 participates in the functionality ofthe traffic distributor 140. This embodiment allows executing severalvirtualized ADC devices in parallel, to execute a single virtualized ADCdevice on several blade servers 310-1 through 310-M, or combinationthereof. Thus, a vADC can be executed in parallel on different bladeservers 310-1 through 310-M.

As a non-limiting embodiment, each of the blade servers 310-1 through310-M acts a virtualized ADC device, such as the device 100 illustratedin FIG. 1. In such an embodiment, the switch 120 selects to which of theblade servers (e.g., to server 310-2) to distribute an incoming traffic.The blade server receiving the incoming traffic forwards it to one ofits vADC instances by means of its traffic distributor. The switch 120selects the destination blade server based on one or more of the VLANtags, MAC addresses, or any other layer-2 parameters designated in theincoming packets. In an embodiment, the switch 320 performs a selectionprocess described in detail below with a reference to FIG. 4.

In accordance with various embodiments disclosed herein, computingresources allocated for a vADC 130-i are defined in terms of an ADCcapacity unit (ADC-CU). An ADC capacity unit is an atomic, notdividable, block that encapsulates various computing resourceparameters. The computing resource parameters include, but are notlimited to, computation resources, memory resources, a number ofconcurrent allowable connections, a configuration limit, a networkbandwidth, a number of new connections per second, storage resources,secure sockets layer (SSL) hardware processing resources, andcompression hardware processing resources. The parameters may be relatedto computing resources that are internal and/or external to thevirtualized ADC device performing the virtualization.

A virtualized ADC device can serve one or more capacity units. With thisaim, the computing resources of the device or array of devices (e.g., amulti-blade system) are equally divided into the number of capacityunits that can be supported by the device. One or more capacity unitscan be allocated to a vADC.

The computation resources parameter defines the number of CPU cores andpart of cores that can run a vADC 130-i. The percentage of memoryresources defines an amount of memory in bytes to be allocated per vADC130-i. The percentage of total device bandwidth defines the percentageof total device bandwidth allocated for a vADC 130-i. If this limit isexceeded on a vADC 130-i, the packet may be dropped. The percentage ofstorage devices parameter defines an amount of storage space in Mbytesthat are available for a vADC 130-i. This parameter includes severalvalues for each system storage device. The concurrent connection limitparameter defines a maximal number of sessions that a vADC 130-i canhandle in parallel. The configuration limit defines a maximal number ofVIPs, servers, and proxy IPs that can be configured per vADC. Thepercentage of secure sockets layer (SSL) capabilities defines the numberof new SSL connections that a vADC 130-i is allowed to process. Thepercentage of compression hardware capabilities parameter defines theamount of bytes that a vADC 130-i is allowed to compress/decompress. Thepercentage of HTTP object caching memory resources defines the number ofbytes that a vADC 130-i is allowed to store in caching memory.

It should be noted that the number of capacity units and percentage ofsystem resources allocated for each ADC capacity unit may vary from onevirtualized ADC device (or a multi-blade system) to another, dependingon the virtualized ADC device and its hardware configuration. Forexample, the amount of memory allocated for each ADC capacity unitdepends on the amount of memory included in a particular device, thenumber of CPU cores, and so on. A virtualized ADC device 100 may offerone or more ADC capacity unit configurations for a selection by a user.For example, one configuration of the device offers a small number oflarge capacity units, and a second configuration of the device offers alarger number of small capacity units. This allows the user of a device100 to upgrade or downgrade the utilization of the computing resources,on-demand. It should be appreciated that the multiple configurationscapacity units for a virtualized ADC device 100 is beneficial in “cloudcomputing applications” where the utilization of resources isdynamically changed. According to one embodiment, each ADC capacity unitconfiguration may be associated with a different price.

A maximum allowable of capacity units (CU_(MAX)) for the virtualized ADCdevice are set, for example, by a system administrator. An ADC capacityunit is defined to include a CU_(MAX)-portion of each resource parameterdefined above. That is, an ADC capacity unit may be defined as follows:

CU={SP _(—) C#/CU _(MAX) ; MEM/CU _(MAX) ; BW/CU _(MAX) ; CONN./CU_(MAX) ; cVIP, cRS, cPIP, STOR/CU _(MAX) ; SSL/CU _(MAX) ; CMP/CU_(MAX)}

where “SP_C#” is the number of available CPU cores, “BW” is the systemmaximal theoretical throughput, “MEM” is the amount of available memory,“CONN” is the maximal number of concurrent connections for the vADC130-i to handle at each point of time, “STOR” is the free space onstorage device(s) available for applications' execution, “SSL” is a SSLHW capacity to open new SSL connections, “CMP” is compression HWthroughput capacity to process incoming byte stream, and cVIP, cRS, cPIPis a maximum number of VIPs, real servers and proxy IP, respectively,that totally should be supported per ADC capacity unit (these numbersare defined by a system administrator).

FIG. 4 is a non-limiting flowchart 400 illustrating a process forcreating a vADC in accordance with an embodiment. As mentioned above, avADC provides the functionality of a physical ADC device deployed in adatacenter. The creation of vADCs is controlled by the management module120. In an embodiment, the creation of vADC 130-i initializes a vADVinstance in the device 100. As mentioned above, multiple vADV instancescan reside and operate in parallel in the device 100.

At S410, the management module 120 tries to allocate the number ofcapacity units requested by a user (e.g., a system administrator) for anewly created vADC. At S420, network parameters, such as a MAC addressand a management IP address, are allocated for a vADC to be created.

At S430, the allocated capacity unit(s) is assigned to a vADC.Specifically, S430 includes selection of CPU cores, or portions thereof,and memory for executing the newly created vADC. In accordance with anembodiment, such selection is performed to achieve a balanced system byminimizing the difference in a number of capacity units running on everyCPU core. The balance is reached by dispersing vADCs across CPU cores.In another embodiment, the selection tries to minimize latency impactcaused by executing several vADCs on the same core, by assigning a CPUcore to a single vADC. Step S430 will be now described in more detailwith reference to FIG. 5.

In a virtualized ADC device 100, every CPU core and memory attached tothis core are initially divided into a predefined number of capacitypartitions (CP). The dimension parameters of capacity partitions (corepercentage and memory size) will be equal to the computation resources,and memory resources parameters in the capacity units. The number ofcapacity partitions coexisting in the device 100 should be equal to thenumber of the maximum capacity units, one partition for every capacityunit. A list of capacity partitions available for allocation on aspecific core is maintained in virtual pools 510 of free capacitypartitions, one virtual pool 510 for every CPU core 520. As depicted inFIG. 5, there are 3 virtual pools 510-1, 510-2, and 510-3 for 3 CPUcores 520-1, 520-2, and 520-3.

The allocation of the requested capacity units to the created vADCincludes selecting on which CPU core or cores and from which memory thenewly created vADC will run. At S501, allocation of capacity partitionsfrom the respective virtual pool 510 of the selected core 520 isperformed. This allows for reserving, on the selected core, a percentageof computation resources and an amount of memory. Then, at S502, the CPUcore (e.g., 520-2), or cores selected to run, is instructed to behave inaccordance to new resource reservation. In an embodiment, the allocationis performed by the management module 120.

Referring back to FIG. 4, at S440, an instantiation of a vADC (e.g.,vADC 130-i) takes place. At S450, the network parameters, ADC capacityunit allocation as well as the assignment information are passed to thecreated vADC. At S460, a unique identification (ID) number is assignedto the vADC. The ID number may be utilized, for example, to communicatewith the vADC.

In accordance with an embodiment of the invention, capacity unitsallocated and assigned to a created vADC can be modified during theruntime of the vADC. That is, one or more capacity units may be added orde-allocated from a running vADC, without halting the vADC. In anotherembodiment, a vADC can be migrated from one virtualized ADC device toanother. As mentioned above, each virtualized ADC device ischaracterized with different computing resources, thus when migratingvADCs between devices, allocated capacity units are convertedaccordingly.

Further, a created instance of a vADC can be deleted when it is nolonger required. The process for deleting a vADC instance can beinitiated by the system administrator. The process includes destroyingthe vADC running one or more CPU cores, and de-allocating the networkingparameters, capacity units, and reserved computing resources.

In order to allow low latency of packets processing the vADCs 130 and toensure that every vADC 130-i can fully utilize the CPU resourcesallocated by its capacity unit(s), a scheduling process is implementedby the traffic distributor 140. The scheduling process schedules betweenseveral vADCs 130-1 through 130-n sharing the same CPU core. vADCsrunning on different CPU cores process packets independently, thusscheduling between such vADCs is not required.

In accordance with an embodiment, a time slice which is the minimum timewindow that a vADC 130-i can receive for utilizing a CPU core is set toan initial value. In an exemplary embodiment, the time slice is set to25 microseconds.

Then, for each vADC to be executed on a CPU core, the process allows theexecution of the vADC during a consecutive number of time slices equalto the number of capacity units for that vADC on that core. Once a vADCconsumes its time slice or slices, the scheduling process providesexecution time for the next vADC in line. The scheduling between vADCsmay be performed in a round-robin manner or any other schedulingalgorithm. In an embodiment, the scheduling process ensures a particularvADC 130-i does not wait more than a predefined time period to receiveexecution time on a CPU core. This waiting time period determines thelatency of the processing tasks by a vADC 130-1, thus the latency of thevirtualized ADC device. It should be appreciated that the latency isconfigurable, and hence can be set to optimally serve network processingtasks performed by the vADC.

An example for the scheduling process is provided in FIG. 6, where 3vADCs 611, 612, and 613 are executed on a single CPU core. The capacityunits allocated for vADCs 611, 612, and 613 are 3, 2, and 1respectively. As can be noticed, vADC 611 is executed during time slicesTS1, TS2, TS3, and TS7, TS8, TS9; vADC 612 runs at time slices TS4, TS5and TS10, TS11; and vADC 613 is executed during time slices TS6 andTS12. The time slice TSO is reserved for the traffic distributor (TD140).

In the example provided in FIG. 6, the maximum latency is 7 time slices.For example, in the worst case illustrated in FIG. 6, the vADC 613 waits6 time slices until it can be executed on a CPU core. The best caseshown in FIG. 6 is for vADC 611 that waits only 4 time slices for itsexecution.

In accordance with an embodiment, during its execution a vADC can borrowadditional resources. This feature allows for the efficient processingof burst traffic. To this end, a pool of resources (e.g., a pool 510-3in FIG. 5) is reserved during the initialization of the virtualized ADCdevice 100. Then, during a runtime of a vADC 130-i, additional resourcescan be temporarily allocated for a configurable limited time period.

In an embodiment, to prevent extensive usage of the reserved pool,statistics are collected about the utilization of the pool of resourcesby each vADC 130-i. vADCs that exceed an allowed quota to borrow areblocked from consumption of the reserved resources and their allocatedcapacity units may be permanently increased. In certain embodiments,statistics collected on the utilization of capacity units and reservedresources can be reported to the user. Using such information the usermay decide to increase/decrease the number of capacity units for a vADCor the number of vADCs executed in the device 100.

FIG. 7 shows a non-limiting flowchart 700 illustrating the vADCselection process performed by the traffic distributor 140, uponreception of a packet, according to an embodiment. Through the selectionprocess, the traffic distributed 140 determines to which of the vADC130-1 through vADC 130-n an incoming traffic should be forwarded forprocessing.

As mentioned above, an association between internal ports and externalports may be performed using either a dedicated port topology or sharedport topology. The dedicated port topology assumes full networkinfrastructure separation between vADCs 130 running on the samevirtualized ADC device 100. In a shared port topology, two or moredifferent vADCs 130 can share the same external port 115, where trafficseparation is performed according to VLAN association and destinationMAC addresses.

At S710, a check is made to determine whether the external port 115 onwhich an incoming packet was received employs a dedicated topology(mode), and if so, execution continues with S720; otherwise, a sharedtopology is employed and execution advances to S730.

At S720, another check is made to determine if the external port 115 isbound to a specific vADC 130. The check may be performed based onlayer-1 parameters (e.g., layer-1 port setting and port aggregation) ofthe incoming packet. If S720 results in an affirmative answer, at S724,the vADC to process the incoming packet is selected using an inputexternal port number. Then, execution advances to S740.

If S720 results with a negative answer, at S726, the vADC to process theincoming packet is selected by matching a VLAN tag embedded in thepackets to the VLAN-to-vADC translation table, which returns a vADCbased on a VLAN tag in the packet. It should be noted that S726 isperformed for all types of packets, e.g., broadcast and unicast packets.It should be further noted that if there is no match of a VLAN tag tovADC, the packet is dropped and execution returns an error message.

At S730, it is checked if the packet destination MAC address is abroadcast MAC address, and if so, at S732, the vADC is selected usingthe VLAN-to-vADC translation table. It should be noted that in thiscase, multiple vADCs can be returned when matching a single input VLANtag. At S734, the packet is cloned as the number of vADCs returned atS732. Thereafter, execution continues with S740. If S730 returns anegative answer, the packet is a unicast packet, and at S736, the VLANtag and destination MAC address are matched against a MAC translationtable, which returns a single vADC to process the incoming packet. Itshould be noted that if the packet includes an unknown destination MACaddress, the packet is dropped and an error message is generated. Thetranslation tables mentioned above are preconfigured or can beconfigured during vADC creation and can be modified by a user (e.g., asystem administrator).

At S740, the traffic distributor 140 has to select one of the one ormore CPU cores belonging to the selected vADC for the execution of thepacket processing. In one embodiment, the selection of the CPU core isbased on any of the layer-2, layer-3, and layer-4 headers of thereceived packet. As an example, a CPU core may be selected bycalculating a hash value of the layer-3 or layer-4 headers of thepacket. Using the computed hash value, the CPU core(s) is selected forthe vADC by considering that on different cores the vADC can usedifferent core share. If the layer-3 (IP) or layer-4 (TCP) parametersare unknown or the packet is the not an IP packet, the packet is sent toa pre-defined core (e.g. core 0). Other embodiments based on functionsand distribution policies to select the CPU core(s) for the vADC will beapparent to one of ordinary skill.

At S750, the packet is sent to the selected vADC and CPU core. It shouldbe noted that layer-1, layer-2, layer-3, and layer-4 mentioned hereinrefer to the layer-1, layer-2, layer-3 and layer-4 defined in the OSImodel, i.e., the physical, MAC and TCP/IP layers. It should be notedthat when a broadcast packet should be sent to more than one vADC, S740and S750 are repeated per each vADC.

FIG. 8 shows a non-limiting and exemplary flowchart 800 illustrating aprocess for routing packets processed by a vADC, as performed by thetraffic distributor 140, according to one embodiment. At S810, it ischecked if the external port on which an original packet was receivedemploys a dedicated topology (mode). If so, execution continues withS820; otherwise, a shared topology is employed and execution advances toS830.

At S820, another check is made to determine if an internal port of avADC that outputs the packet is bounded to a specific external port 115.The check may be performed based on the layer-1 and/or layer-2parameters (e.g., VLAN tag) designated in the packet. If S820 resultswith an affirmative answer, at S822, the external port for the packet isselected using a vADC-to-port translation table, which returns a portnumber of an external port (port 115 FIG. 1) to send the packet. Then,execution continues with S849. If S820 results in a negative answer, atS824, the external port 115 which sends out the packet is selected usinga VLAN-to-port translation table, which returns a single port number,based on the VLAN tag in the packet. Then, execution continues withS849.

At S830, it is checked if the packet transmission between vADCs isallowed. If so, at S840, it is further checked if the packet'sdestination MAC address is a broadcast MAC address. If so, at S842, theexternal port is selected using the VLAN-to-port translation table.Then, at S843, the VLAN tag of the packet is matched against theVLAN-to-vADC translation table to determine if the packet should be sentto other vADCs, and if so, which one. At S844, the packet is cloned asthe number of vADCs matching the VLAN tag. Then, execution continueswith S860.

If S840 returns a negative answer, the packet is a unicast packet, andat S846, the destination MAC address is searched in a MAC translationtable, which returns a destination port (which may be either internal orexternal) according to the input MAC address. At S848, it is checked ifthe type of the port is an internal, and if so, execution continues withS860; otherwise at S849, the packet is sent to the external port.

At S860, a destination internal port is selected based, in part, on thedestination IP address of the packet. At S870, the packet is sent to thevADC with the selected internal port.

If S830 results with a negative answer, execution continues with S850where the destination MAC address of the packet is searched in a MACtranslation table, which returns a destination external port accordingto the input destination MAC address. Then, the packet is sent to theexternal port. The translation tables mentioned above, with reference toflowchart 800, are preconfigured and can be modified by a user (e.g., asystem administrator).

In an embodiment, reception and transmission of packets is performedusing a zero copy mechanism implemented by the traffic distributor 140.This mechanism allows packet reception and transmission from/to externalinterfaces and packet switching between vADC's, without transferring thepackets among the virtualized ADC device, and cloning or copying thepackets. With this aim, the packets are saved in a shared memory thatcan be accessed by the vADCs 130 and traffic distributor 140. Thus,instead of, for example, cloning a packet, a pointer to a shared memoryis provided to each vADC that should process the packets.

FIG. 9 shows an exemplary diagram utilized to describe a process forindependently executing vADCs having different application versionsaccording to certain embodiments disclosed herein. As depicted in FIG.9, over an operating system 900, two vADCs 910 and 920, a managementmodule 930, and a traffic distributor 940 are executed. It should benoted that for the sake of brevity only two vADCs are shown in FIG. 9,however, the teaching disclosed herein can be applied to any number ofvADCs greater than 2.

The operating system 900 may be, but is not limited to, a Windows-basedoperating system, a Linux based operating system, a UNIX based operatingsystem, VXwork6, and the like. Each of the vADCs 910 and 920 performsthe network processing tasks as discussed in greater detail above, forexample, with respect to vADCs 130. Each of the vADCs 910 and 920 is aself-contained application that includes all resources, such asexecutable code, configuration, user privileges, to be independentlyexecuted over the operating system 900. Specifically, each of the vADCs910 and 920 does not run its own operating system (e.g., a guestoperation system) and it is not created or managed by the operatingsystem 900.

Each of the vADCs 910 and 920 runs a different application version, eachof which is differentiated by its own version of executable code and insome cases configuration. As a result, the vADCs 910 and 920 may performdifferent network processing tasks, support a different set ofprocessing features, and so on. In addition, one vADC may run an updatedor newer application version relative to the other vADC. For example, avADC 920 runs a newer application version of the same version as thevADC 910, but the newer version contains fixed bugs identified in theolder version.

The management module 930 and traffic distributor 940 are processes thatmanage and schedule the execution of the vADCs 910 and 920 as describedin detail above. According to this embodiment, the management module 930and the traffic distributor 140 are configured to enable independency ofexecution of the vADCs 910 and 920. Specifically, the management module930 manages the application versions of the vADCs in the virtualized ADCdevice. With this aim, the management module 930 maintains, at least theapplication versions supported by the virtualized vADC device and aversion number of each application version. The version informationrelated to a vADC is gathered by the management module 930 during thecreation of the vADC or once the vADC is created. The versioninformation may be utilized by the traffic distributor 940 forcommunication with the vADCs 910 and 920 and by a system administrator.In one embodiment, the version information for each vADC being executedis displayed through a control panel, e.g., in a form of a GUI.

As mentioned above, traffic can be forwarded from one vADC to another.As each of the vADCs 910 and 920 are of different application versions,each may utilize different data structures or formats to encapsulate thetraffic and other information. However, all vADCs should recognize andinterpret the received traffic regardless of the data structure it isincluded in. In addition, the traffic distributor 940 also encapsulatesthe traffic and other information required for the traffic processing ina data structure that may be illegible to the vADC 910. To overcome thisproblem, it is important to maintain consistency in the communicationbetween the vADCs 910 and 920. With this aim, the traffic distributor940 maintains consistency in the communication between the vADCs 910 and920.

In one embodiment, the traffic distributor 940 communicates with thevADCs 910 and 920 using a virtualization communication protocol (VCP).The VCP defines a set of data structures and rules for exchanginginformation among the vADCs as well as between vADCs and the trafficdistributor. In another embodiment, the traffic distributor 940 isprogrammed to recognize the different data structures of the differentapplication versions of the vADCs. A data structure that includestraffic, e.g., from vADC 910 and directed to, e.g., vADC 920 thatutilizes a different data structure is received by the trafficdistributor 940, which modifies the received data structure to a formatreadable by the vADC 920. Then, the traffic distributor 940 sends themodified data structure to the vADC 920.

In accordance with one embodiment, a hardware infrastructure of avirtualized ADC device maintains multiple software images, each of whichrepresents a different application version of a vADC. For example, thehardware infrastructure maintains software images of vADCs 910 and 920.In addition, the hardware infrastructure maintains one or more softwareimages of the traffic distributor and management module.

The virtualized ADC device is installed with only one version of thetraffic distributor and management module. However, a user (e.g., asystem administrator) can select which application version of a vADC torun on the virtualized device. Based on the user selection, themanagement module decides which version of the vADC to create and run.The process for creating a vADC is described in detail above. Theversion information associated with the newly created instance of a vADCis gathered and shared with the distribution module.

The embodiments of independent execution of vADCs can be furtherutilized in order to enable software upgrades without affecting clientsutilizing the services of vADCs and reducing the overall downtime of adatacenter. As schematically illustrated in FIG. 10, a virtualized ADCdevice 1000 is deployed in a datacenter that serves clients 1021, 1022,1023, and 1024. The datacenter may be part of an infrastructure thatprovides cloud computing services.

The virtualized ADC device 1000 executes vADCs 1031 and 1032, each ofwhich runs a different application version. As a non-limiting example,traffic from clients 1021, 1022 is handled by the vADC 1031 and trafficfrom clients 1023, 1024 is handled by the vADC 1032. When a newerversion of a vADC, e.g., a vADC 1033, is introduced, an image of thisversion is stored in the virtualized ADC device 1000. The user canselect to create an instance of the vADC 1033. For example, the vADC1033 may be a software update of the vADC 1031. Once created the vADC1033 is executed independently of the vADCs 1031 and 1032. The clients1021, 1022, 1023, and 1024 may migrate to a new vADC, but this isperformed without halting the operation of any of the vADCs. Forexample, traffic from clients 1021 and 1024 can now be directed to thevADC 1033, for example, as the vADC 1033 provides traffic processingservices not supported by the other vADCs. Again, it is important toemphasize that during the creation of the vADC 1033 and the migration ofclients 1021 and 1024, the vADCs 1031 and 1032 do not halt theiroperation, and the creation of the vADC 1033 is transparent to theclients. Thus, the device 1000 and the datacenter do not need to be shutdown when new application versions of a vADC should be installed in thedatacenter.

FIG. 11 is a non-limiting and exemplary flowchart 1100 illustrating amethod for executing vADCs of different application versions by avirtualized ADC device according to one embodiment. At S1110, at leasttwo vADCs each of which has a different application version relative tothe other are created by a management module (e.g., module 120). Asmentioned above, a software image of each of the vADCs is stored in thevirtual ADC device, and the creation of the vADC is performed uponselection of the user. The process for creating a vADC is described indetail above with respect to FIG. 4. A vADC created by the managementmodule is a self-contained application that includes all resources, suchas executable code, configurations, and user privileges. The vADC doesnot run any operation system.

At S1120, the version information associated with the created vADCs iscollected and transferred to the traffic distributor. The versioninformation includes at least a version number, configuration, theformat of the internal communication protocol (e.g., a protocol versionutilized for communication with other vADCs), the date the vADC wascreated, the date the vADC was installed, and so on.

At S1130, the created vADCs are independently executed over a singleoperating system of the virtualized ADC device. The execution of thevADCs is managed and controlled by the traffic distributor. As notedabove, during the execution of the vADCs, additional new vADCs havingdifferent application versions may be created and executed over theoperating system of the computing device.

The foregoing detailed description has set forth a few of the many formsthat the invention can take. It is intended that the foregoing detaileddescription be understood as an illustration of selected forms that theinvention can take and not as a limitation to the definition of theinvention.

Most preferably, the various embodiments of the invention can beimplemented as any combination of hardware, firmware, and software.Moreover, the software is preferably implemented as an applicationprogram tangibly embodied on a program storage unit or computer readablemedium. The application program may be uploaded to, and executed by, amachine comprising any suitable architecture. Preferably, the machine isimplemented on a computer platform having hardware such as one or morecentral processing units (“CPUs”), a memory, and input/outputinterfaces. The computer platform may also include an operating systemand microinstruction code. The various processes and functions describedherein may be either part of the microinstruction code or part of theapplication program, or any combination thereof, which may be executedby a CPU, whether or not such computer or processor is explicitly shown.In addition, various other peripheral units may be connected to thecomputer platform such as an additional data storage unit and a printingunit. Furthermore, a non-transitory computer readable medium is anycomputer readable medium except for a transitory propagating signal.

1. A method for executing virtual application delivery controllers(vADCs) having different application versions over a computing device,comprising: installing a virtualization infrastructure in the computingdevice; creating by the virtualization infrastructure a plurality ofvADCs having different application versions, wherein each vADC iscreated from a software image maintained in a hardware infrastructure ofthe computing device; gathering version information associated with eachof the plurality of vADCs; independently executing the plurality ofvADCs over an operating system of the computing device; and controllingthe execution of the plurality of the vADCs over an operating system ofthe computing device using the virtualization infrastructure using inpart the version information.
 2. The method of claim 1, wherein each ofthe plurality of vADCs is a self-contained application that includes atleast executable code, configuration files, and user privilegesprogrammed for its respective application version.
 3. The method ofclaim 2, wherein each of the plurality of vADCs is programmed to performnetwork processing tasks including at least one of: load balancing ofthe incoming traffic, compression of the incoming traffic, and cachingof the incoming traffic, wherein each vADC performs a network processingtask defined by its respective application version.
 4. The method ofclaim 2, wherein each of the plurality of vADCs does not execute its ownguest operating system.
 5. The method of claim 1, wherein the versioninformation of each of the vADCs includes at least one of a versionnumber, configuration of the vADC, and a protocol version utilized forcommunication with other vADCs, and a creation date.
 6. The method ofclaim 1, wherein the plurality of vADCs created by the virtualizationinfrastructure are selected by a user.
 7. The method of claim 1, whereinthe virtualization infrastructure includes: a management moduleprogrammed to create the plurality of the vADCs and gather at least theversion information for each of the plurality of vADCs; and a trafficdistributor programmed to distribute incoming traffic to one of theplurality of vADCs and schedule the execution of the plurality of vADCson one or more core processors in the hardware infrastructure.
 8. Themethod of claim 7, wherein the traffic distributor further allowsconsistency in the communication between the plurality of vADCs.
 9. Themethod of claim 8, further comprises: recognizing a communication formatemployed by a source vADC and directed to a destination vADC using thesource and destination vADCs; determining if the communication formatemployed by the source vADC is recognizable by the destination vADCs;and manipulating the communication format employed by the source vADC tomeet the communication format of the destination vADC, wherein thesource vADC and the destination vADC are part of the plurality of vADCs.10. The method of claim 9, wherein creating a vADC of the plurality ofvADCs further comprises: allocating at least one ADC capacity unit tothe vADC; assigning at least one network parameter to the vADC;assigning the least one ADC capacity unit to the vADC; instantiating thevADC from a software image related to an application version of thecreated vADC, the software image is maintained in the hardwareinfrastructure; and assigning a unique identification (ID) number to thevADC.
 11. The method of claim 10, wherein the at least one networkparameter is at least a medium access network (MAC) address.
 12. Themethod of claim 11, wherein the at least one ADC capacity unit is anatomic block encapsulating a plurality of hardware resource parametersrelated to hardware resources of the hardware infrastructure.
 13. Themethod of claim 11, wherein the hardware resource parameters include atleast one core processor, or portion thereof, a throughput of a networkinterface, and a percentage of memory.
 14. The method of claim 12,wherein the hardware resource parameters further include at least oneof: a number of concurrent connections, storage space on a storagedevice, a number of secure sockets layer (SSL) connections, acompression throughput capacity to process the incoming traffic, anumber of virtual IP addresses, a real IP addresses, and a proxy IPaddresses.
 15. The method of claim 10, wherein the vADCs is furtherassigned with one more ADC capacity unit configurations, wherein each ofthe ADC capacity unit configurations includes a different number ofcapacity units that can be allocated to the vADC.
 16. The method ofclaim 1, further comprises: creating at least one more vADC having anapplication version different than the application versions of theplurality of vADCs, wherein the creation of the at least one more vADCis performed during the execution of the plurality of vADCs withouthalting their execution.
 17. A non-transitory computer readable mediumhaving stored thereon instructions for causing one or more processingunits to perform a process for executing virtual application deliverycontrollers (vADCs) having different application versions over acomputing device, comprising: installing a virtualization infrastructurein the computing device; creating by the virtualization infrastructure aplurality of vADCs having different application versions, wherein eachvADC is created from a software image maintained in a hardwareinfrastructure of the computing device; gathering version informationassociated with each of the plurality of vADCs; independently executingthe plurality of vADCs over an operating system of the computing device;and controlling the execution of the plurality of vADCs over anoperating system of the computing device using the virtualizationinfrastructure using in part the version information.
 18. Thenon-transitory computer readable medium of claim 17, wherein each of theplurality of vADCs is a self-contained application that includes atleast executable code, configuration files, and user privilegesprogrammed for its respective application version.
 19. Thenon-transitory computer readable medium of claim 17, wherein each of theplurality of vADCs does not execute its own guest operating system. 20.The non-transitory computer readable medium of claim 17, furthercomprises: creating at least one more vADC having an application versiondifferent than the application versions of the plurality of vADCs,wherein the creation of the at least one more vADC is performed duringthe execution of the plurality of vADCs without halting the execution ofthe plurality of vADCs.